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Abstract 

There  is  a  growing  interest  for  the  integration  of  critical  thinking,  defined  as  the  capacity  of  thinking  about 
one’s  own  reasoning,  into  military  practices.  In  this  paper,  we  discuss  the  potential  of  critiquing  systems 
(critics)  -  software  programs  that  provide  a  critique  of  the  user-generated  solution  -  for  training  critical 
thinking  skills. 

More  specifically,  we  discuss  the  use  of  two  types  of  critics,  generic  and  experiential,  which  respectively 
use  doctrine-related  and  case-related  knowledge.  The  generic  critic  applies  general  knowledge  about 
standard  practices,  while  the  experiential  critic  makes  the  practitioner  consider  distributional  data,  as 
provided  by  case  bases  and  lessons  learned.  This  paper  discusses  the  type  of  issues  that  can  thus  be 
addressed. 


1.  Introduction 

The  aim  of  the  present  paper  is  to  show  the  usefulness  of  critiquing  systems  (critics)  for 
supporting  critical  reasoning  in  the  military  context.  We  argue  here  that  these  advisory 
systems  can,  on  the  one  hand,  provide  generic  critiques  that  would  remind  the  user  of 
general  doctrine-related  knowledge,  and  on  the  other  hand,  bring  him,  by  means  of 
experiential  critics,  to  consider  relevant  knowledge  retrieved  from  similar  cases  and 
lessons  learned. 

The  paper  is  organized  as  follows:  first,  we  outline  the  principles  of  critical  thinking  as 
discussed  in  relevant  literature  (Section  2),  and  then  we  compare  the  dynamics  of 
collaborative  criticism  dialogues  in  the  human-critic  setting  with  the  internalized  dialogue 
of  critical  reasoning  (Section  3).  In  Section  4,  we  discuss  the  type  of  judgment  biases  that 
the  use  of  critics  can  correct.  Next,  we  show  how  generic  critics  can  guide  the  user  in  his 
problem  solving  process  by  reminding  him  of  relevant  issues  and  factors  (Section  5)  and 
how  the  experiential  critic,  using  case-based  reasoning  and  lessons  learned,  can  make  the 
user  consider  or  leam  from  previously  experienced  cases  (Section  6). 


2.  Critical  thinking 


The  subject  of  critical  thinking  has  been  studied  for  years  (e.g.  in  philosophy, 
psychology,  education)  and  has  recently  gained  importance  in  the  military  domain,  in 
particular  for  training  military  decision-makers  dealing  with  complex  and  knowledge- 
intensive  tasks.  Challenges  that  face  military  decision-makers,  in  the  context  of 
increasing  complexity,  changing  character  of  military  operations,  information  overload, 
increased  responsibilities  and  initiatives,  reinforce  the  usefulness  of  thinking  critically. 

The  roots  of  critical  thinking  are  as  ancient  as  the  vision  of  Socrates,  known  as  “Socratic 
questioning”,  who  established  the  importance  of  asking  deep  questions  that  probe 
profoundly  into  thinking  before  we  accept  ideas  as  worthy  of  belief.  He  established  the 
importance  of  seeking  evidence,  closely  examining  reasoning  and  assumptions,  analyzing 
basic  concepts,  and  tracing  out  implications  not  only  of  what  is  said  but  of  what  is  done 
as  well.  In  his  mode  of  questioning,  Socrates  highlighted  the  need  in  thinking  for  clarity 
and  logical  consistency  [1].  Later,  the  military  strategist  Carl  von  Clausewitz  also 
promoted  the  value  of  critical  thinking  for  strategic  leaders  in  his  writings  [2]. 

Several  definitions  of  critical  thinking  have  been  developed  by  contemporary  theorists 
that  highlight  self-examination  of  thinking.  Freeley  and  Steinberg  [3]  define  critical 
thinking  as  “the  ability  to  analyze,  criticize,  and  advocate  ideas;  to  reason  inductively  and 
deductively;  and  to  reach  factual  or  judgmental  conclusions  based  on  sound  inferences 
drawn  from  unambiguous  statements  of  knowledge  or  belief’.  A  critical  thinking  attitude 
involves  asking  questions,  defining  a  problem,  examining  evidence,  analyzing 
assumptions  and  biases,  avoiding  oversimplification,  considering  other  interpretations, 
and  tolerating  ambiguity. 

Thinking,  to  be  critical,  must  be  analyzed  and  assessed  according  to  intellectual  standards 
and  criteria,  such  as  clarity,  accuracy,  relevance,  depth,  breadth,  and  logicalness.  The 
process  of  thinking  critically  entails  the  examination  of  elements  of  thought  implicit  in  all 
reasoning:  purpose,  problem  or  question  at  issue,  assumptions,  concepts,  empirical 
grounding,  reasoning  leading  to  conclusions,  implications  and  consequences,  objections 
from  alternative  viewpoints,  and  frame  of  reference  [4].  From  these  elements,  Col. 
Guillot  [5]  highlights  some  thinking  skills  that  are  particularly  relevant  in  the  military 
context.  A  military  strategist  must  assess  if  his  purpose  is  in  line  with  goals  and  needs, 
look  at  situations  from  multiple  points  of  view,  change  focus  and  shift  his  thinking  to  see 
things  differently. 

Halpern  [6]  presents  critical  thinking  as  a  goal-directed  process  as  it  focuses  on  desired 
outcome.  Cohen  [7]  also  conceives  of  critical  thinking  as  a  dialogue  that  explores 
alternative  possibilities  under  the  constraints  of  context-specific  goals.  He  describes 
critical  thinking  skill  as  ‘asking  and  answering  critical  questions  about  alternative 
possible  state  of  affairs,  to  the  extent  that  such  questioning  is  likely  to  increase  the 
reliability  of  the  overall  activity  in  achieving  its  purpose’. 


Critical  thinking  can  be  introduced  in  the  military  decision  making  realm  for  different 
purposes  (situation  assessment,  mission  analysis,  operation  planning,  etc.).  However, 
there  are  good  reasons  to  believe  that  decision  makers  will  not  engage  in  critical  scrutiny 
of  their  decisions  and  plans  under  stressful  and  time-pressed  conditions.  In  attempts  to 
obtain  the  degree  of  knowledge  needed  to  anticipate  alternative  outcomes,  the  decision¬ 
maker  is  likely  to  be  overwhelmed  by  ‘information  inundation’,  which  can  be  quite  as 
incapacitating  as  the  lack  of  information.  Ill-structured  problems,  uncertainty,  shifting 
goals,  action/feedback  loops,  time  stress,  high  stakes  and  multiple  players  are  other 
factors  [8]  that  affect  decision  making  performance  in  contexts  such  as  the  military 
environment.  Because  of  all  these  difficulties,  adopting  a  normative  decision  making 
approach  would  already  place  huge  demands  on  decision  makers’  resources  and  mental 
capabilities.  Adopting  a  critical  attitude  would  be  an  even  greater  challenge. 

Our  contention  is  that  critical  thinking  skills  can  be  inculcated  to  decision  makers,  by  the 
use  of  critiquing  systems  during  training  sessions,  and  that  this  exercise  will  eventually 
lead  to  better  decision  making  in  real  situations  of  analysis,  evaluation  and  planning. 

3.  Dialogues  with  critiquing  systems 

Critiquing  systems  are  a  class  of  program  that  receive  as  input  the  statement  of  the 
problem  and  the  user-proposed  solution  and  produce  as  output  a  critique  of  the  user’s 
judgment  and  knowledge  [9].  The  feedback  (criticism)  output  to  the  user  may  report 
errors,  point  out  incompleteness,  suggest  alternatives,  or  offer  heuristic  advice.  As  such, 
critiquing  systems  present  a  great  potential  for  supporting  critical  reasoning. 

These  advisory  systems  exemplify  the  dynamics  of  critical  discussions  in  which  a 
proponent  defends  a  claim  against  challenges  of  various  kinds  by  an  opponent  or  critic 

[10] .  This  type  of  interaction,  be  it  in  a  human-human  or  human-system  setting,  is  a 
promising  framework  for  supporting  critical  thinking,  primarily  because  of  the 
‘functional  similarity  between  rationally  persuading  another  individual  to  accept  or  reject 
a  position,  and  rationally  determining  for  oneself  whether  a  position  is  acceptable  or  not’ 

[11] . 

The  internalized  dialogue  of  critical  thinking  where  the  same  person  embodies  two 
distinct  participants,  can  be  transposed  to  the  critiquing  systems’  paradigm,  viewed  as  a 
‘mutual  exchange  of  viewpoints  between  the  expert  human  and  the  expert  critic’  [12]. 
Fischer  et  al.  [13]  define  the  interaction  with  a  critiquing  system  as  ‘a  dialogue  in  which 
the  interjection  of  a  reasoned  opinion  about  a  product  or  action  triggers  further  reflection 
on  or  changes  to  the  artefact  being  designed’. 

During  the  criticism  dialogues,  the  user  constructs  partial  solutions  that  the  critic 
evaluates  with  respect  to  a  set  of  criteria  and  constraints.  The  output,  signaling  errors  and 
deficiencies,  feeds  back  into  the  user’s  mental  model  and  results  in  elaboration  or 
modification  of  certain  elements.  The  system  thus  helps  the  user  validate  a  body  of 
knowledge  through  a  collaborative  dialogue  with  the  aim  of  improving  the  product  and 
enhancing  the  quality  of  decision  making.  Criticism  dialogues  have  many  positive 


outcomes  such  as  error  elimination,  deeper  reflection,  better  situation  analysis,  and 
growth  of  knowledge  [13]. 


4.  Errors  and  Biases 

Competency  in  critical  thinking  helps  individuals  to  distinguish  fact  from  judgment  or 
belief  from  knowledge.  One  of  the  main  objectives  of  critiquing  systems  is  to  prevent  or 
correct  erroneous  decision  making  due  to  lack  or  misuse  of  knowledge,  poor  heuristics  or 
judgment  biases  [14]. 

In  [14],  Silverman  presents  an  error  model  which  accounts  for:  (1)  errors  in  the  expert’s 
knowledge  base  due  to  missing  concept  or  missing  knowledge,  and  (2)  errors  in  the 
expert’s  reasoning  processes  due  to  cognitive  bias  or  systematic  selection  of  poor 
judgement  heuristics. 

Reason  [15]  refers  to  the  same  categories  of  errors  as  knowledge-based  and  rule-based 
mistakes.  For  Reason,  knowledge-based  mistakes  have  their  roots  in  two  aspects  of 
human  cognition:  bounded  rationality  and  the  fact  that  knowledge  relevant  to  the  problem 
space  is  nearly  always  incomplete  and  often  inaccurate.  Rule -based  mistakes,  on  the  other 
hand,  are  either  due  to  misapplication  of  good  rules  or  to  the  application  of  bad  rules. 
Reason  associates  these  mistakes  to  problem  solvers’  habit  of  pattern-matching.  If  a 
pattern  is  recognized  and  some  kind  of  match  is  performed,  a  previously  established  if 
(condition)  then  (action)  rule-based  solution  is  applied.  Only  when  this  procedure  fails  to 
provide  an  adequate  solution,  will  they  move  to  the  mode  of  making  inferences  from 
knowledge-based  mental  models  of  the  problem  space.  With  his  Recognition-Primed 
Decision  Model,  Klein  [16]  shows  that  this  type  of  decision  making  prevails  in  time- 
compressed  environments.  Experienced  people  make  rapid  decisions  using  situation 
assessment  to  generate  a  workable  course  of  action  and  mental  simulation  to  evaluate  that 
course  of  action. 

Although  these  early  judgments  about  a  problem  help  decision  makers  to  save  time, 
reduce  complexity  and  rapidly  generate  partial  solutions,  they  are  often  based  on  informal 
reasoning.  Silvennan  talks  of  mental  myopia  due  to  force  of  habit,  selective  perception, 
and  availability  biases  [14].  He  gives  examples  of  this  in  the  domain  of  Anny 
forecasting:  for  anny  forecasters,  past  examples  of  degradation  and  damage  from  threats 
they  encountered  and  studied  in  their  own  discipline  are  the  most  vivid  and  easy  to  recall. 
For  example,  attack  helicopter  specialists  most  vividly  visualize  potential  damage  from 
enemy  attack  helicopters  and  not  the  damage  resulting  from  encounters  with  other 
threats. 

Unaware  of  their  judgment  biases,  problem  solvers  and  planners  tend  to  be  overconfident 
in  evaluating  the  correctness  of  their  knowledge.  They  will  tend  to  justify  their  chosen 
course  of  action  by  focusing  on  evidence  that  favors  it  and  by  disregarding  contradictory 
signs.  Also  known  as  the  confirmation  bias,  this  strategy  consists  of  seeking  information 
consistent  with  current  beliefs  and  avoiding  falsifying  evidence. 


In  their  study  on  critical  reasoning  biases,  Klaczynski  et  al.  [17]  also  show  that  when 
presented  with  pertinent  information,  individuals  apply  a  wide  range  of  reasoning  tactics 
to  preserve  the  integrity  of  their  goals  and  beliefs.  They  readily  assimilate  belief- 
enhancing  information  without  engaging  in  critical  scrutiny,  but  they  engage  in  extensive 
data  analysis  and  deep  processing  when  facing  belief-threatening  information.  The  use  of 
sophisticated  strategies  in  the  latter  case  can  result  in  refutation  of  the  data. 

For  Silverman,  the  role  of  the  critic  in  this  context  is  to  draw  the  practitioner’s  attention 
to  two  categories  of  knowledge  that  are  critical  to  successful  task  outcomes.  This  includes 
‘overlooked  knowledge’  that  the  human  ignores  for  reasons  of  biased  judgment  and 
‘missing  knowledge’  that  the  practitioner  is  unaware  of  due  to  incomplete  or  out-of-date 
training. 

We  will  argue  here  that  in  the  C2  domain,  the  critiquing  system  can  intervene  by 
providing  general  doctrine-related  knowledge  that  the  practitioner  ‘normally  masters  but 
has  neglected  to  apply’  [12],  as  well  as  experiential  knowledge  that  is  based  on  other 
individuals’  experiences  and  observations  and  which  are  either  unknown  to  the  user  or 
are  not  accessible  during  his  problem  solving  task. 

5.  Use  of  doctrine-related  knowledge  for  critiquing 

A  critiquing  system  can  have  ‘generic  critics’  that  provide  general  knowledge  about 
standard  practices  and  ‘experiential  critics’  that  relate  the  current  problem  solving 
situation  to  past  experiences  and  lessons  learned. 

The  generic  critic,  following  what  is  traditionally  called  the  analytic  approach  to 
critiquing,  can  walk  the  user  through  his  problem  solving  task  by  reminding  him  of 
certain  rules,  criteria  and  constraints.  Such  a  critic  detects  error  occurrences  or 
deficiencies  that  it  then  turns  into  assistance  opportunities  [18].  It  relies  on  the  user’s 
proposed  plan  of  action  sequences  (or  focus  of  attention)  as  a  basis  for  the  generation  of 
critiques  and  the  critiques  it  formulates  are  based  on  domain  knowledge. 

A  generic  critic  can  check  whether  certain  procedures  have  been  correctly  followed  (e.g. 
Operation  Planning  Procedure),  whether  the  requirements  formulated  have  been  met  (e.g. 
Mission  Order)  and  whether  mandatory  constraints  for  the  achievement  of  specified  goals 
are  fulfilled. 

Within  the  analytic  framework,  critics  do  not  need  to  have  a  complete  understanding  of 
the  product.  Because  their  role  is  not  to  offer  a  solution,  but  to  check  the  user’s  partial 
solution  for  flaws,  these  critics  are  well-suited  for  vague  and  ill-defined  problem  domains 
where  they  can  provide  support  by  applying  general  knowledge.  In  fact,  critiquing 
systems  using  this  mechanism  can  be  very  successful  in  application  domains  in  which 
problem  solving  is  performed  on  a  constraint-satisfaction  basis.  Miller  [19],  who  was  the 
first  researcher  to  see  a  great  potential  in  the  use  of  critiquing  systems  for  decision 
support  observed  that  critiquing  systems  are  most  appropriate  for  tasks  that  require  that 


the  practitioner  remember  or  consider  a  lot  of  information  such  as  the  procedures,  risks, 
benefits,  side  effects  and  costs. 

Thus,  in  high-level  tasks  of  decision  making  and  planning,  the  generic  critic  can  help  the 
user  test  different  options  against  a  whole  array  of  requirements  and  constraints.  In  such 
sufficiently  structured  framework,  critics  can  check  that  the  norms  and  standards  have 
been  respected,  that  there  are  no  obvious  inconsistencies  or  errors  and  that  problematic 
conditions  do  not  hold. 

Most  of  the  military  applications  of  critiquing  systems  use  this  critiquing  procedure  to 
perform  COA  evaluation  and  planning.  As  part  of  the  HPKB  and  RKF  DARPA 
programs,  whose  primary  objectives  were  to  provide  tools  facilitating  the  building  of 
large  knowledge  based  systems  by  domain  experts,  a  number  of  COA  critiquing  systems 
were  conceived  that  make  use  of  knowledge  input  by  subject  matter  experts  as  well  as 
ontologies  (common  domain  theories)  on  planning.  SHAKEN  uses  a  set  of  rules  input  by 
subject  matter  experts  to  control  the  user’s  action  and  enables  him  to  simulate  a  COA  in 
order  to  evaluate  its  performance  [20].  Disciple-COA  is  another  critiquing  tool  which 
uses  previously  learned  rules  (from  examples  provided  by  experts)  to  assess  the  viability 
and  correctness  of  a  course  of  action,  highlight  its  strengths  and  the  weaknesses  with 
respect  to  principles  of  war  and  tenets  of  army  operations,  justify  the  assessments  made 
and  propose  improvements  to  the  COA  [21,  22],  Inspect  [23]  is  an  evaluation  tool  that 
checks  the  consistency  of  air  campaign  plans  (completeness,  feasibility)  and  alerts  the 
user  about  inconsistencies  and  potential  problems.  In  the  collaborative  decision  making 
context  of  ROLF  2010  [24]  (C2  system  research  project  conducted  by  the  Swedish 
National  Defense  College),  the  critiquing  system  provides  feedback  on  the  effects  of  staff 
members’  plans  on  each  other’s  intentions. 

The  only  military  application  aimed  at  critical  thinking  training  that  we  know  of  is  a 
simulation  tool  proposed  by  Cohen  and  Shastri  [25].  A  connectionist  network  is  used  to 
model  rapid  reflexive  processes  (rapid  generation  of  coherent  interpretations  and  plans) 
and  the  reflective  processes  that  monitor  and  regulate  them. 

Less  ambitiously,  we  think  that  critiquing  systems  can  use  different  intervention 
strategies  (proactive,  reactive)  to  trigger  further  reflection  on  the  different  elements  that 
lead  to  a  given  decision.  In  [26],  Silverman  emphasizes  the  role  of  preventive  critics 
which  can  warn  experts  about  nonregressive  and  overconfidence  biases  and  explain  how 
to  avoid  them.  Placed  before  the  task,  these  ‘influencers’  divulge  common  errors  and 
remind  the  user  of  the  kind  of  decisions  that  are  expected  before  the  user  begins  the 
problem  solving  process. 

For  example,  at  the  strategic/operational  level,  critics  can  bring  the  decision  maker  to 
question  some  of  the  main  issues  which  will  guide  further  actions.  This  can  be  done  by 
means  of  leading  question  asking.  Following  the  different  steps  in  decision  analysis 
process,  the  critic  can  make  the  decision  maker  verify  whether  the  problem  has  been 
correctly  identified,  that  he  is  pursuing  the  right  objectives  and  that  they  have  been 
clearly  articulated,  that  all  the  options  have  been  considered,  that  the  criteria  used  to 


evaluate  them  are  indeed  the  most  relevant,  that  the  implementation  of  an  option  does  not 
imply  undesired  consequences. 

As  Col.  Guillot  [5]  writes,  “the  critical  thinker  must  ask:  what  is  my  real  purpose,  what  is 
the  key  issue,  what  is  the  most  relevant  information,  what  are  the  correct  concepts  in  this 
case,  are  my  assumptions  valid,  have  I  drawn  the  correct  inferences,  what  poin  ts  of  view 
matter,  and  what  are  my  desired  implications ?”  Illustrating  the  fourth  point,  he  rightly 
observes  that  one  of  the  lessons  learned  after  September  1 1  is  that  an  aircraft  can  be  used 
as  a  flying  weapon.  ‘On  that  day  the  concept  of  ‘a  missile’  or  ‘bomb’  changed  and  so  did 
our  idea  of  how  to  protect  against  such  a  conceptual  shift’.  Our  conceptual  model  can 
undergo  less  radical  yet  decisive  changes  when  confronted  with  belief-threatening 
information. 

Critiquing  systems  can  exploit  intelligence  data  sources  to  present  possibilities  that  the 
decision  maker  had  not  foreseen.  Such  influencing  strategies  can  be  complemented  by 
persuasive  techniques  such  as  explanation  and  argumentation.  We  think  that  the 
critiquing  system  must  also  account  for  cases  where  the  user  himself  takes  the  initiative 
of  submitting  his  opinion  to  critical  scrutiny.  This  can  be  a  request  for  a  punctual 
evaluation  or,  as  we  later  discuss,  a  demand  for  presentation  of  cases  with  similar 
problems  or  problem  elements. 

Criticism  dialogues  can  be  more  efficient  if  carried  out  by  a  critic  than  by  oneself, 
because  the  system  can  present  more  problem  solving  approaches  that  one  can  possibly 
think  of.  Also,  system-generated  critiques  run  a  smaller  risk  of  being  rejected  since  as 
Fischer  et  al.  [27]  point  out,  critiques  coming  from  an  intelligent  computer  assistant 
might  be  more  tolerable  than  critiquing  from  a  human,  because  they  are  handled  as  a 
private  matter  between  the  user  and  the  computer.  The  other  advantages  are  related  to  the 
training  context.  First,  the  user  will  be  less  bothered  by  the  intrusiveness  of  critics  in  this 
setting.  Second,  in  a  learning  position,  the  practitioner  will  tend  to  be  more  open  to  other 
perspectives  and  will  not  resort  to  heuristics,  which  are  normally  used  for  reasons  of  time, 
habit,  stress,  or  perceived  efficiency.  Finally,  he  will  probably  tend  to  evaluate  data 
independently  from  his  personal  goals  and  beliefs,  which  as  Klaczynski  et  al.  [17]  have 
claimed,  are  indispensable  for  effective  critical  thinking. 

6,  Use  of  experiential  knowledge  for  critiquing 

‘Critical  thinking  is  the  intellectually  disciplined  process  of  processing  infonnation 
gathered  from,  or  generated  by,  observation,  experience,  reflection,  reasoning,  or 
communication,  as  a  guide  to  belief  and  action’  [4],  From  this  perspective,  critical 
reasoning  consists,  at  least  in  part,  of  considering  acquired  experiences  and  observations 
and  relating  them  to  the  current  problem  solving  situation. 

In  his  attempt  to  correct  judgment  biases  by  means  of  critiquing  systems,  Silverman  also 
observes  that  the  critic  must  help  decision  makers  regress  to  distributional  data 
(Kahneman  and  Tversky  [28]),  which  is  knowledge  about  the  distribution  of  outcomes  in 
similar  situations. 


Experiential  critics  can  fulfill  this  requirement  by  relating  the  user’s  decision  situation  to 
similar  cases  experienced  by  his  co-workers.  When  dealing  with  complex  and 
knowledge-intensive  problems  where  the  problem-solving  process  requires  exploring  a 
large  problem  space,  and  when  this  problem  solving  enviromnent  is  not  time-constrained, 
it  may  be  beneficial  for  decision-makers  to  learn  from  others’  experiences  in  addition  to 
relying  on  their  own  memory.  The  analysis  and  interpretation  of  these  experiences  can 
help  him  figure  out  if  (and  how)  the  problem-solving  processes  and  outcomes  can  be 
reused,  or  adapted  to  the  current  situation. 

Two  related  topics  are  of  interest  for  providing  relevant  experiential  knowledge  for 
problem  solving,  namely  the  case-based  reasoning  paradigm  and  lessons  learned 
management.  These  are  in  line  with  the  knowledge  management  vision  that  expert 
knowledge  (or  in  general  corporate  knowledge)  should  be  carefully  captured  and 
organized  in  order  to  facilitate  its  subsequent  exploitation  by  knowledge  workers.  It  also 
relates  to  the  “learning  organization”  concept  that  refers  to  an  organization’s  capability  to 
gain  insight  and  understanding  from  experience  through  experimentation,  observation, 
analysis,  and  a  willingness  to  examine  both  successes  and  failures. 

6.1.  Case-based  reasoning 

The  case-based  reasoning  (CBR)  paradigm  [29]  is  based  on  the  premise  that  human 
beings  use  analogical  reasoning  or  experiential  reasoning  to  leam  and  solve  complex 
problems.  Instead  of  relying  solely  on  general  knowledge  of  a  problem  domain,  or 
making  associations  along  generalized  relationships  between  problem  descriptors  and 
conclusions,  CBR  is  able  to  utilize  the  specific  knowledge  of  previously  experienced, 
concrete  problem  situations  (cases).  It  is  also  an  approach  to  incremental,  sustained 
learning,  since  a  new  experience  is  retained  each  time  a  problem  has  been  solved,  making 
it  immediately  available  for  future  problems.  Cases  usually  consist  of  information  about 
the  situation,  the  solution,  results  of  that  solution  and  key  attributes  that  can  be  used  for 
quickly  searching  for  similar  patterns  of  attributes.  A  new  problem  is  solved  by  retrieving 
one  or  more  previously  experienced  cases,  reusing  the  case(s)  in  one  way  or  another, 
revising  the  solution  based  on  reusing  a  previous  case,  and  retaining  the  new  experience 
by  incorporating  it  into  the  existing  knowledge-base  (case-base).  New  solutions  are 
generated  by  retrieving  the  most  relevant  cases  and  adapting  them  to  fit  new  situations. 

One  advantage  of  CBR  is  that  inexperienced  people  can  draw  on  the  knowledge  of 
experienced  colleagues  to  solve  their  own  problems.  Thus,  the  experiential  critic  that  uses 
CBR  can  act  as  a  recommender  system,  or  a  training  tool. 

In  the  military  domain,  case-based  reasoning  approach  has  been  exploited  for  situation 
assessment,  planning  or  decision  aiding.  In  planning,  in  particular,  it  has  been  used  for 
retrieving  and  reusing  previous  plans  or  plans’  elements  from  previously  solved  problems 
(case  library). 


Battle  planner  [30],  one  of  the  first  CBR  systems  in  the  military  domain,  is  a  case-based 
retrieval  system  for  battle  planning  that  projects  the  effects  of  plans  based  on  experiences 
stored  in  a  case  library.  The  user  can  examine  among  the  best-matching  cases  which 
cases  were  successful  and  which  were  considered  as  failures  by  experts’  evaluations, 
compared  to  the  proposed  plan.  Cases  can  be  used  to  criticize  and  repair  plans,  the  user 
can  perform  what-if  analyses  with  the  objective  to  improve  an  initial  solution. 

JADE  [31]  is  a  planning  tool  that  has  been  applied  to  military  force  deployment  in  order 
to  retrieve  and  reuse  force  modules  (FMs)  from  previous  plans  whose  force  capabilities 
and  composition  satisfy  the  current  situation. 

HICAP  [32]  is  a  general  purpose  planning  architecture  that  integrates  a  conversational 
case-based  planner  NaCoDAE  to  interactively  refine  a  task  into  an  operational  plan.  Its 
application  to  noncombatant  evacuation  operations  (NEO)  is  both  doctrine  and  case- 
driven.  Whereas  doctrine  describes  general  aspects  of  planning,  experiences  from 
previous  operations  suggest  more  detailed  information  suitable  for  the  current 
environment. 

One  of  the  challenges  for  building  CBR  applications  is  the  representation  of  cases  and  the 
identification  of  features  that  best  characterize  them  in  order  to  facilitate  the  retrieval  of 
best-matching  cases  in  a  new  situation.  Traditionally,  cases  are  expressed  as  a  problem 
description  and  solution  pair.  In  order  to  exploit  CBR  as  a  critiquing  process,  cases 
should  encapsulate  sufficient  information  about  the  context/situation,  as  well  as 
arguments  for  the  solution  proposed,  or  knowledge  about  the  problem-solving  process 
(e.g.  decision  choices,  and  justifications). 

6.2.  Lessons  Learned 

In  the  Command  and  Control  domain,  Lessons  Learned  constitute  a  fundamental 
knowledge  asset.  As  part  of  its  traditional  culture,  military  people  have  continuously 
reported  observations  or  lessons  after  operations  or  exercises.  Past  operations’  lessons 
convey  important  experiential  knowledge  (both  about  successes  and  failures)  that  can  be 
learned,  reused  or  avoided  in  future  similar  situations  in  a  way  to  make  the  best  decisions 
and  undertake  the  best  actions  (e.g.  preparation  of  future  operations,  selecting  best  course 
of  actions,  etc.).  A  definition  provided  in  [33]  states  that  a  lesson  is  a  validated 
experiential  knowledge  from  a  work  experience.  It  represents  tacit  knowledge,  either 
positive  or  negative  that  can  be  reused  (i.e.  retrieved  and  applied  during  subsequent 
problem-solving)  to  improve  a  targeted  organizational  process  by  suggesting  a  relevant 
contribution  to  a  work  practice. 

As  an  example,  The  ALDS  system  (Active  Lessons  Delivery  System)  [33]  brings  lessons 
to  the  attention  of  users  when  applicable  to  the  user’s  current  decision-making  task  under 
similar  conditions.  A  case-based  reasoning  approach  is  used  to  match  the  context  of  the 
current  situation  with  relevant  lessons  stored  in  the  system.  In  another  military  context, 
the  ACPA  (Air  Campaign  Planning  Advisor)  system  [34]  includes  a  corporate  memory 


containing  video-clips  in  which  experts  relate  their  experience  that  can  support  users  in 
their  tasks  through  a  model-based  task  tracking  system. 

Lessons  Learned  do  not  provide  a  solution  to  a  problem  to  be  solved  but  rather  provide 
relevant  elements  that  may  support  or  impede  the  reasoning  process,  thus  influencing  the 
outcome. 

6.3.  Experiential  critics 

The  experiential  critic  can  provide  knowledge  specific  to  situations  which  share  certain 
characteristics  with  cases  already  experienced  or  lessons  learned,  and  which  are  stored  in 
the  system’s  knowledge  base.  The  exploitation  of  cases  and  lessons  learned  as  a 
knowledge  asset  aims  at  drawing  both  from  successes  (i.e.  what  went  well)  as  well  as 
from  failures  in  order  not  to  repeat  errors  twice.  The  reuse  of  this  knowledge  by  a 
critiquing  system  aims  at  making  recommendations  from  a  past  situation  that  apply  in  the 
current  situation. 

Experiential  critics  would  provide  knowledge  which  is  not  part  of  the  doctrine  or  the 
usual  training  of  practitioners  and  which  is  essentially  acquired  through  experience.  For 
example,  important  information  about  the  new  environments  (remote  countries  with 
different  social,  religious  and  cultural  backgrounds)  in  which  the  armed  forces  are 
brought  to  operate,  can  only  be  transferred  to  newcomers  through  the  experience  of  their 
co-workers.  Along  this  perspective,  Davis  and  Fu  [35]  have  recently  proposed  to  gather 
culturally  specific  experiential  data  from  available  sources  in  order  to  design  a  cultural 
decision  aid  tool. 

In  unfamiliar  situations,  experiential  critics  can  use  past  observations  to  remind  the  user 
of  relevant  facts  and  hypotheses  that  he  may  have  neglected  or  that  he  is  unaware  of.  For 
example,  the  critic  can  remind  the  operator  that  certain  resources  will  be  necessary  if  the 
operations  have  to  take  place  in  a  specific  region.  It  can  draw  the  attention  of  the  user  on 
factors  he  may  have  overlooked  or  whose  importance  has  been  over  or  under-estimated  in 
the  current  situation.  It  can  make  the  user  focus  on  the  implications  of  certain  choices  or 
the  effects  that  certain  factors  may  have  on  the  evaluation  of  other  factors.  Finally,  it  can 
suggest  alternatives  for  certain  important  assumptions  on  which  later  decisions  may 
depend. 

As  discussed  previously,  in  complex  decision-making  contexts,  individuals  employ 
subjective  probabilities  based  on  their  degree  of  belief  in  the  possible  outcomes  and 
resort  to  the  use  of  heuristics.  Heuristics  allow  humans  to  quickly  process  huge  amounts 
of  information  and  often  arrive  at  reasonable  decisions  under  difficult  circumstances. 
Using  heuristics  implies  a  schema-driven  reasoning  [36]  where  the  individuals  use  their 
prior  knowledge  and  experience  in  the  decision  episode  domain  to  make  inferences  about 
a  situation  and  decide  what  infonnation  is  valuable  for  the  solution.  This  ‘enables  speedy 
assessment,  search,  selection  and  interpretation’  [8]  of  information. 


Such  reasoning,  where  perceived  information  fits  into  an  existing  schema  may  be  prone 
to  many  judgment  biases,  as  we  mentioned  before.  We  can  speculate  that  in  specific 
instances,  when  the  decision  maker  is  placed  in  an  unfamiliar  environment,  the  use  of 
heuristics  may  represent  an  even  more  serious  source  of  error.  An  experiential  critic  can 
both  remind  a  decision  maker  facing  a  familiar  context  not  to  overlook  data  that  are  not 
integrated  into  his  schema  of  the  decision  situation,  and  help  a  decision  maker  facing  an 
unfamiliar  situation  take  advantage  of  his  co-workers’  knowledge  and  experience. 

The  provision  of  experiential  knowledge  in  an  unfamiliar  context  is  aimed  at  avoiding 
that  the  practitioners  rely  on  their  own  experience.  Recent  studies  have  questioned 
experts’  abilities  in  non-repeated  decision  tasks  [37].  These  works  have  revealed  that 
experience  improves  decision-making  only  for  well-practiced  or  anticipated  tasks  with 
established  procedures,  while  on  the  other  hand,  in  unique  or  unanticipated  tasks, 
experience  may  actually  contribute  to  poorer  decision  making.  Explanations  for  this  poor 
decision  performance  of  experts  are  the  rigidity  and  abstraction  of  the  expert’s  mental 
models  and  his/her  complacency  and  overconfidence. 

Within  the  less  stressful  context  of  training,  experiential  critics  can  target  confirmation 
and  availability  biases  by  presenting  similar  cases  with  different  decision  outcomes,  but 
they  can  also  fill  the  gaps  of  missing  knowledge,  by  providing  the  users  with  field 
experience  missing  from  their  usual  training. 

Contrary  to  critics  that  look  for  error  occurrences  or  deviations,  these  critics  would  hint  at 
experiential  knowledge  whenever  available.  Naturally,  the  practitioner  cannot  request  for 
this  knowledge  if  he  is  unaware  that  it  exists.  By  providing  contextualized  access  to  this 
type  of  infonnation,  critics  reduce  the  cost  of  learning  and  optimize  the  effort-accuracy 
trade-off  involved  in  accessing  domain  knowledge. 

However,  case-related  information  must  be  made  available  at  different  levels  of  detail. 
For  example,  the  detection  of  similarities  along  certain  dimensions  (e.g.  area  of 
operations,  type  of  events,  type  of  threat,  etc.)  would  activate  punctual  critiques  that 
would  be  related  to  those  same  aspects.  Knowing  that  these  critiques  are  based  on  cases 
or  lessons  learned,  the  user  can  decide  to  obtain  more  in-depth  infonnation  by  viewing 
the  whole  scenario  or  situation.  Thus,  the  user  can  proceed  to  further  examination  of 
cases  (for  example,  through  hypertext  links)  in  order  to  contextualize  those  aspects  for 
which  observations,  advice,  positive  or  negative  critiques  are  provided.  By  looking  at 
commonalities  and  differences  of  retrieved  situations,  users  analyze  the  context  of  these 
situations,  assess  if  they  fit  with  their  understanding  of  the  current  situation,  and 
determine  to  what  extent  retrieved  experiences  could  apply,  and  if  not  why. 

The  obvious  consequence  of  this  process  is  learning,  and  the  expected  one  is  learning  to 
think  critically.  The  critic  does  not  only  provide  the  user  with  knowledge  of  past 
experiences,  helping  him  leam  about  relevant  criteria  and  issues,  but  it  also,  in  the  long 
tenn,  enrols  him  into  the  habit  of  gathering  knowledge  and  considering  different 
perspectives  before  committing  to  a  decision. 


7.  Conclusion 


In  the  preceding,  we  outlined  the  principles  of  critical  thinking  and  discussed  the  ways  in 
which  they  can  be  supported  by  critiquing  systems.  Critiquing  systems  do  not  empower 
the  user  with  the  meta-cognitive  capacity  of  evaluating  his  own  reasoning,  but  can  give 
him  cues  which  can  make  him  explore  new  perspectives  or  seek  knowledge. 

We  discussed  the  concurrent  use  of  two  types  of  critics,  generic  critics,  which  critique  the 
user’s  solution  with  respect  to  doctrine-related  knowledge,  and  experiential  critics,  which 
provide  relevant  information  retrieved  from  past  cases  and  lessons  learned.  Generic 
critics  can  act  as  cognitive  assistants  that  can  help  the  user  manage  multiple  parameters 
and  apply  domain  knowledge,  but  they  can  also  force  second  thought  by  questioning  the 
user’s  position  and  assumptions.  Experiential  critics,  on  the  other  hand,  can  draw  the 
user’s  attention  on  alternatives  he  had  not  foreseen  and  acquaint  him  with  knowledge  he 
does  not  possess.  These  feedbacks  enhance  the  user’s  understanding  of  the  problem  and 
may  either  lead  him  to  focus  on  or  to  reconsider  certain  aspects  of  his  solution.  The 
critiquing  process  can  thus  improve  the  quality  of  the  user’s  decision  making  by 
promoting  reflection  and  give  him  the  opportunity  to  learn. 
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Critical  Thinking 


Critical  thinking  is  the  ability  to  think  about  one’s 
thinking  in  such  a  way  as  to  recognize  its  strengths 
and  weaknesses  and,  as  a  result,  to  recast  the 
thinking  in  an  improved  form. 

Needed  because  of: 


Increasing  complexity 
Changing  character  of  military  operations 
Information  overload 
Increased  responsibilities 
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Critical  thinking  in  the  military  context 


Asking  questions 

Clearly  defining  the  problem 

Seeking  &  examining  evidence 

Closely  examining  reasoning  and  assumptions 

Analyzing  basic  concepts 

Avoiding  oversimplification 

Considering  alternative  viewpoints 


Tracing  out  implications  and  consequences 
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Critiquing  systems 


•  A  class  of  program  that  receive  as  input  the 
statement  of  the  problem  and  the  user-proposed 
solution  and  produce  as  output  a  critique  of  the 
user’s  judgment  and  knowledge. 

•  Feedback  to  user: 


Report  errors 
Point  out  incompleteness 
Suggest  alternatives 
Offer  heuristic  advice 
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Dialogue  paradigm 
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Critiquing  process 


Defence  R&D  Canada 


R  &  D  pour  la  defense  Canada 


Errors  &  Biases 


•  Knowledge  errors  due  to  missing  concept  or 
missing  knowledge 


•  Errors  in  reasoning  processes  due  to  cognitive  bias 
or  systematic  selection  of  poor  judgment  heuristics 

-  Force  of  habit 


Selective  perception 
Availability  bias 

Confirmation  bias:  seeking  information 
consistent  with  current  beliefs  and  avoiding 
falsifying  evidence. 
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Overlooked  and/or  missing  knowledge 


Critic  can  provide  (in  training  sessions): 


•  Doctrine-related  knowledge  that  the  practitioner 
‘normally  masters  but  has  neglected  to  apply’. 


Experiential  knowledge  that  is  based  on  other 
individuals’  experiences  and  observations  and 
which  are  either  unknown  or  not  accessible  to  the 


user. 
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Generic  critics 


Provide  general  knowledge  about  standard  practices 

Remind  certain  rules,  criteria,  constraints 

Check  whether  procedures  have  been  followed  & 
requirements  have  been  met 


Constraint-satisfaction  problem-solving 

Manage  multiple  factors  (procedures,  risks,  benefits, 
side-effects,  cost) 

Check  for  inconsistencies  and  problematic  conditions 

Check  whether  alternative  solutions  have  been 
considered 
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Generic  critics:  strategies 


Leading  Question  Asking,  Explaining,  Arguing 

-  Problem  has  been  correctly  identified 

-  Right  objectives 

-  Objectives  have  been  clearly  articulated 
All  options  have  been  considered 
Relevant  criteria 
Undesired  consequences 
Valid  assumptions 
Correct  inferences 
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training  context 


•  Different  problem  solving  approaches 

•  Tolerance  to  system-generated  critiques 

•  Tolerance  to  intrusiveness 

•  Openness  to  different  perspectives 

•  Evaluation  of  data  independently  from  personal 
goals  and  beliefs 
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Experiential  critics 


•  Relate  past  experiences  and  observations  to  the 
current  problem-solving  situation. 

•  Make  the  user  regress  to  distributional  data 
(knowledge  about  distribution  of  outcomes  in 
similar  situations) 


Case-based  reasoning  & 

Lessons  learned  management 

Capture  and  organization  of  knowledge  for  future 
use 


An  approach  to  incremental,  sustained  learning 
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Case-Based  Reasoning 


•  Supports  analogical  reasoning. 


•  Use  of  specific  knowledge  of  previously 
experienced,  concrete  problem  situations  as 
opposed  to  making  associations  along  generalized 
relationships. 


Detailed  information  that  can  support 
inexperienced  individuals. 

Constitutes  corporate  memory. 
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Lessons  Learned 


•  Lesson:  ‘A  validated  experiential  knowledge  from 
a  work  experience.’ 

•  Tacit  knowledge  that  can  be  reused  to  improve  a 
process  by  suggesting  a  relevant  contribution  to  a 
work  practice. 


Allows  evaluation  of  both  successes  and  failures. 

Not  solutions  to  problems  but  relevant  elements 
that  may  support  or  impede  a  reasoning  process. 
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Experiential  critics:  strategies 


•  Critic  uses  knowledge  assets  (CBR,  LL)  to  make 
recommendations  proactively. 

•  Acquaint  the  practitioner  with  knowledge  he  is  unaware  of. 
Provide  knowledge  at  different  levels  of  description. 

-  Critic  hints  at  information  based  on  detected  similarities 
(area  of  operation,  type  of  threat,  etc.). 

-  User  proceeds  to  further  examination  of  cases,  thus 
contextualizing  advice,  observation,  positive  or  negative 
critiques. 

-  User  determines  to  what  extent  retrieved  experience  can 
apply. 
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Experiential  critics 


•  Provide  specific  knowledge  that  can  only  be 

acquired  through  the  experience  of  co-workers  (e,g. 
information  about  remote  countries  with  different 
social,  religious  and  cultural  backgrounds). 
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Experiential  critics 


•  Avoid  use  of  heuristics  (schema-driven  reasoning) 
in  unfamiliar  situations. 

1 .  Familiar  situation:  Not  to  overlook  data  that  are 
not  integrated  into  his  schema  of  decision 
situation. 

2.  Unfamiliar  situation:  Take  advantage  of  his  co¬ 
workers’  knowledge  and  experience. 

(Personal  experience  improves  decision-making  only  for 
well-practiced  or  anticipated  tasks  with  established 
procedures) 
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Experiential  critics 


•  Operate  both  on  errors  of  knowledge  and  errors  of 
reasoning. 

-  Target  confirmation  bias  and  availability  biases 
(similar  situations  but  different  outcomes) 


-  Fill  the  gaps  of  knowledge  (missing  from  usual 
training) 

Reduce  cost  of  learning 
Learn  to  think  critically 
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Conclusion 


•  Provide  ‘unsuspected’  knowledge; 


•  Make  the  user  consider  relevant  criteria  &  issues; 

•  Make  the  user  question  position  and  assumptions; 


Draw  the  user’s  attention  on  alternatives  he  had  not 
foreseen; 

Promote  reflection  &  create  learning  opportunities; 

Enroll  the  practitioner  into  the  habit  of  gathering 
data  and  considering  different  perspectives  before 
committing  to  a  decision. 
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